New Convergence Properties of the Primal Augmented Lagrangian Method

and Applied Analysis 3 Given x, λ, μ, c , the augmented Lagrangian relaxation problem associated with the augmented Lagrangian L is defined by min L ( x, λ, μ, c ) s.t. x ∈ Ω. Lλ,μ,c Given ε ≥ 0, then the ε-optimal solution set of Lλ,μ,c , denoted by S∗ λ, μ, c, ε , is defined as { x ∈ Ω | Lx, λ, μ, c ≤ inf x∈Ω L ( x, λ, μ, c ) ε } . 2.2 IfΩ is closed and bounded, then the global optimal solution of Lλ,μ,c exists. However, if Ω is unbounded, then Lλ,μ,c maybe unsolvable. To overcome this difficultly, we assume throughout this paper that f is bounded on Ω from below, that is, f∗ : inf x∈Ω f x > −∞. 2.3 This assumption is rathermild in optimization programming, because otherwise the objective function f can be replaced by e x . It ensures that the ε-optimal solution set with ε > 0 always exists, since L x, λ, μ, c is bounded from below by 2.1 and 2.3 . Recall that a vector x∗ is said to be a KKT point of P if there exist λi ≥ 0 for each i 1, . . . , m and μj for each j 1, . . . , l such that 0 ∈ ∇f x m ∑ i 1 λi∇gi x l ∑ j 1 μj∇hj x NΩ x∗ , λi gi x∗ 0, ∀i 1, . . . , m, 2.4 whereNΩ x∗ denotes the normal cone ofΩ at x∗. The collection set of all λ∗ and μ∗ satisfying 2.4 is denoted by Λ x∗ . The multiplier algorithm based on the primal augmented Lagrangian L is proposed below. One of its main features is that the Lagrangianmultipliers associatedwith equality and inequality constraints are not restricted to be bounded, whichmakes the algorithm applicable for many problems in practice. Algorithm 2.1 Multiplier algorithm based on L . Step 1. Select an initial point x0 ∈ R, λ0 ≥ 0, μ0 ∈ R, c0 > 0, and ε0 ≥ 0. Set k : 0. Step 2. Compute λ 1 i max { 0, ckgi ( x ) λki } , ∀i 1, . . . , m, 2.5 μ 1 j μ k j ckhj ( x ) , ∀j 1, . . . , l, 2.6 4 Abstract and Applied Analysis εk 1 εk k 1 , 2.7 ck 1 ≥ k 1 max ⎧ ⎨ ⎩ 1, m ∑ i 1 ( λ 1 i )2 , l ∑ j 1 ( μ 1 j )2 ⎫ ⎬ ⎭ . 2.8 Step 3. Find x 1 ∈ S∗ λk 1, μ 1, ck 1, εk 1 ; Step 4. If x 1 ∈ X and λ 1, μ 1 ∈ Λ x 1 , then STOP; otherwise, let k : k 1 and go back to Step 2. The iterative formula for εk 1 given in 2.7 is just used to guarantee its convergence to zero. In fact, in the practical numerical experiment, we can choose εk 1 εk/ck to improve the convergence of the algorithm. The following lemma gives the relationship between the penalty parameter ck and the multipliers λ and μ. Lemma 2.2. Let λ, μ, ck be given as in Algorithm 2.1, then the following terms λ ck , μ ck , ( λ )2 ck , ( μ )2 ck 2.9 all approach to zero as k → ∞. Proof. This follows immediately from 2.8 . For establishing the convergence property of Algorithm 2.1, we first consider the perturbation analysis of P . Given α ≥ 0, define the perturbation of feasible region as X α { x ∈ Ω | gi x ≤ α, ∣hj x ∣ ≤ α, i 1, . . . , m, j 1, . . . , l, 2.10 and the perturbation of level set as L α { x ∈ Ω | f x ≤ v 0 α. 2.11 It is clear that X 0 coincides with the feasible set of P . The corresponding perturbation function is given as v α inf { f x | x ∈ X α . 2.12 The following result shows that the perturbation value function is upper semicontinuous at zero. Lemma 2.3. The perturbation function v is upper semicontinuous at zero from right. Proof. SinceX 0 ⊂ X α for any α ≥ 0, then v α ≤ v 0 by definition 2.12 . This implies that lim supα→ 0 v α ≤ v 0 . Abstract and Applied Analysis 5 Lemma 2.4. Let λ, μ, ck, εk be given as in Algorithm 2.1. For any ε > 0, one has S∗ ( λ, μ, ck, εk ) ⊆ { x ∈ Ω | L ( x, λ, μ, ck ) ≤ v 0 ε } , 2.13and Applied Analysis 5 Lemma 2.4. Let λ, μ, ck, εk be given as in Algorithm 2.1. For any ε > 0, one has S∗ ( λ, μ, ck, εk ) ⊆ { x ∈ Ω | L ( x, λ, μ, ck ) ≤ v 0 ε } , 2.13 whenever k is sufficiently large. Proof. For any given ε, it follows from 2.7 and Lemma 2.4 that when k is large enough, we have 1 2ck m ∑ i 1 ( λki ck )2 1 2ck l ∑ i 1 ( μki ck )2 εk ≤ ε. 2.14 Therefore, for x ∈ S∗ λ, μ, ck, εk , L ( x, λ, μ, ck ) ≤ inf { L ( x, λ, μ, ck ) | x ∈ Ω } εk ≤ inf { L ( x, λ, μ, ck ) | x ∈ X 0 } εk ≤ inff x | x ∈ X 0 } 1 2ck m ∑ i 1 ( λki ck )2 1 2ck l ∑ i 1 ( μki ck )2 εk ≤ v 0 ε. 2.15 Lemma 2.5. Let λ, μ, ck be given as in Algorithm 2.1. For any ε > 0, one has { x ∈ Ω | Lx, λ, μ, ck ) ≤ v 0 ε ⊆ X ε . 2.16 whenever k is sufficiently large. Proof. We prove this result by the way of contradiction. Suppose that we can find an ε0 > 0 and an infinite subsequence K ⊆ {1, 2, . . .} such that z ∈ x ∈ Ω | Lx, λ, μ, ck ) ≤ v 0 ε, ∀k ∈ K, 2.17 but z / ∈ X ε0 , ∀k ∈ K. 2.18 It follows from 2.17 that v 0 ε ≥ L ( z, λ, μ, ck ) f ( z ) ck 2 ⎡ ⎢ ⎣ l ∑ j 1 ⎛ ⎝hj ( z ) μkj ck ⎞ ⎠ 2 m ∑ i 1 max { 0, gi ( z ) λki ck }2 ⎤ ⎥ ⎦. 2.19 Since z / ∈ X ε0 , it needs to consider the following two cases. 6 Abstract and Applied Analysis Case 1. There exist an index j0 and an infinite subsequence K0 ⊆ K such that |hj0 z | > ε0. It then follows from 2.19 that v 0 ε ≥ f∗ ck 2 ⎡ ⎢ ⎣ l ∑ j 1 ⎛ ⎝hj ( z ) μkj ck ⎞ ⎠ 2 m ∑ i 1 max { 0, gi ( z ) λki ck }2 ⎤ ⎥ ⎦ ≥ f∗ ck 2 ⎛ ⎝hj0 ( z ) μkj0 ck ⎞ ⎠ 2 . 2.20 Using Lemma 2.2 and the fact that |hj0 z | ≥ ε0 gives us ⎛ ⎝hj0 ( z ) μkj0 ck ⎞ ⎠ 2 ≥ 1 2 ε0, 2.21 whenever k is sufficiently large. This, together with 2.20 , yields v 0 ∞ by taking k ∈ K0 approaching to ∞, which leads to a contradiction. Case 2. There exist an index i0 and an infinite subsequence K0 ⊆ K such that gi0 z > ε0. It follows from 2.19 that v 0 ε ≥ f∗ ck 2 ⎡ ⎢ ⎣ l ∑ j 1 ⎛ ⎝hj ( z ) μkj ck ⎞ ⎠ 2 m ∑ i 1 max { 0, gi ( z ) λki ck }2 ⎤ ⎥ ⎦